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What  does  the  Department  of  Defense  (DOD)  community  know  about  the  effectiveness 
of  the  Commander’s  Emergency  Response  Program  (CERP),  how  to  use  it  to  greatest 
effect,  and  cost  versus  its  value  to  operations?  DOD  has  been  employing  this  “weapons 
system”  for  over  a  decade,  and  so  much  has  been  written  on  the  topic  that  one  would  think  quite  a 
lot  should  be  known.  In  fact,  the  surface  of  understanding  has  barely  been  scratched. 

Claims  regarding  CERP  effectiveness  are  wide  ranging  and  include  arguments  that  it  is  inef¬ 
fective  and  increases  instability  or  that  it  is  effective  only  for  short-term  stabilization.1  Proponents 
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of  these  perspectives  base  their  conclusions  on 
anecdotes,  surveys,  case  studies,  and/or  statis¬ 
tical  analyses  of  large  datasets.  The  argument 
presented  in  this  article  is  that  CERP  has  yet  to 
be  studied  in  a  manner  that  permits  confidence 
in  any  conclusions  presented  to  date. 

Analyzing  a  capability’s  effectiveness  in  a 
complex,  dynamic  environment  is  challenging. 
Compounding  that  challenge  with  ill-defined  and 
a  priori  measures,  or  incomplete  and  in  some  cases 
inconsistent  data  or  data  access  constraints,  dra¬ 
matically  increases  the  burden  on  analysts.  Today’s 
environment  demands  a  research  approach  and 
strong  research  designs  structured  to  minimize  the 
constraints  and  maximize  the  opportunities  pre¬ 
sented.  The  measures  and  data  challenges  require 
a  significant  effort  on  several  fronts:  understanding 
desired  effects,  what  data  are  appropriate,  where 
those  data  reside,  obtaining  access  to  those  data, 
and  “cleaning”  the  data  to  support  analysis.  Is  such 
a  significant  effort  warranted  to  better  inform  the 
debate?  Absolutely. 

Importance  of  the  Debate 

CERP  has  been  in  the  commander’s  tool¬ 
kit  for  years.  It  has  been  employed  for  many 
types  of  projects  large  and  small  in  both  rural 
and  urban  environments.  Since  its  inception 
in  2003,  DOD  has  requested,  and  the  Congress 
obligated,  $6  billion  for  the  program  in  Iraq  and 
Afghanistan.  The  United  States  and  its  coali¬ 
tion  partners  will  continue  reducing  both  forces 
and  funds  for  kinetic  and  nonkinetic  opera¬ 
tions  in  Afghanistan,  thereby  making  a  better 
understanding  of  the  effectiveness  of  CERP 
all  the  more  critical.  Moreover,  over  the  next 
several  years,  in  a  future  security  environment 
that  includes  operations  in  which  population 
engagement  is  an  important  component,  com¬ 
manders  will  likely  expect  CERP  to  be  available 
to  support  their  campaign  plans  and  operations.2 


The  history  of  CERP  is  reason  enough  to  war¬ 
rant  an  effort.  When  combined  with  consider¬ 
ations  of  near-term  and  future  operations,  there 
is  little  room  to  argue  against  a  sustained  com¬ 
mitment  to  rigorous  analyses. 

Lessons  Learned 

With  its  first  appropriation  in  2003,  CERP 
could  have  been  characterized  as  a  Quick 
Reaction  Capability  (QRC),  which  by  its 
nature  is  often  fielded  in  ongoing  operations 
without  predeployment  training  or  training 
materials  and  without  doctrinal  guidance  on  its 
most  effective  employment.3  Such  materials  do 
tend  to  emerge  from  lessons  learned  in  opera¬ 
tions;  however,  those  lessons — and  the  materi¬ 
als  that  follow — are  often  based  on  anecdotal 
evidence.4  In  some  cases,  this  is  understandable, 
as  applying  the  rigorous  test  and  evaluation 
regime  used  for  cost-equivalent  major  defense 
acquisition  programs  is  not  feasible  given  the 
nonpermissive  operational  environment  in 
which  QRCs  tend  to  be  employed.  Rigorously 
evaluating  such  capabilities  in  a  counterinsur¬ 
gency  environment  requires  another  approach. 

Over  the  past  year,  we  have  been  laying 
the  foundation  for,  and  begun  executing,  stud¬ 
ies  on  CERP  and  other  nonkinetic  programs 
and  activities.  Through  these  efforts,  a  num¬ 
ber  of  serious  issues  have  been  identified  that 
should  be  taken  into  account  before  the  defense 
community  can  state  with  confidence  that  it 
understands  the  effectiveness  of  CERP.  Such 
understanding  is  necessary  to  develop  train¬ 
ing  materials,  implement  CERP  to  the  greatest 
effect  in  current  operations,  and  value  it  against 
other  capabilities  in  current  and  forecasted 
operations  in  order  to  inform  doctrine,  concept 
development,  and  programmatic  decisions. 

Tales  of  caution  and  optimism  are  offered 
in  the  sections  that  follow.  Caution  is  expressed 
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most  often  in  reference  to  having  too  much 
confidence  in  the  findings  of  studies  that  have 
been  done  to  date.  Optimism  is  offered  because 
the  serious  issues  can  be  addressed — several 
studies  are  presented  as  examples — thereby 
increasing  the  confidence  that  DOD  and  oth¬ 
ers  can  have  in  decisions  regarding  the  employ¬ 
ment  and  funding  of  CERP. 

Understanding  the 
Independent  Variable 

What  is  the  size  of  the  population  of  proj¬ 
ects?  For  CERP  projects  in  Afghanistan,  the 
focus  of  the  analytical  efforts  discussed  in  this 
article,  data  are  entered  into  three  separate  data¬ 
bases — initially  as  an  Afghanistan  Development 
Report  record,  again  with  additional  data  and 
supporting  media  files  in  Combined  Information 
Data  Network  Exchange  (CIDNE),5  and  finally 
as  a  CERP  Checkbook  entry.6  Working  with  data 
from  2008  to  2010,  it  became  clear  that  all  three 
databases  required  thorough  review  in  order  to 
build  an  accurate  and  comprehensive  account 
of  each  project’s  vital  statistics,  including  but 
not  limited  to  subprojects,  start/end  dates,  loca¬ 
tions,  cost,  type,  and  desired  effects.  Consider, 
for  example,  the  fact  that  the  number  of  CERP 
records  in  the  CIDNE  database  does  not  equal 
the  number  of  CERP  projects.  Many  records  (for 
example,  those  in  the  category  of  Bulk  Funds) 
often  represent  several  projects  that  are  visible 
only  upon  review  of  attached  media  files,  the 
details  of  which  are  not  accounted  for  when 
exporting  unclassified  CERP  data  from  CIDNE. 
This  brings  to  the  fore  the  first  cautionary  note: 
studies  that  have  been  done  to  date  based  on 
unclassified  data  have  in  all  probability  drawn 
conclusions  from  an  incomplete  dataset. 

The  critical  need  to  use  classified  data  in 
CERP  analyses  will  be  a  theme  throughout 
this  article.  In  spite  of  the  best  efforts  of  some 


talented  analysts,  no  comprehensive  study  of 
CERP  in  which  DOD  could  have  confidence  is 
possible  using  only  unclassified  data. 

What  is  the  substance  of  the  population 
of  projects?  As  several  analysts  have  described, 
CERP  projects  are  varied  by  category  (20),  loca¬ 
tion,  date,  duration  (days  to  years),  and  cost. 
Within  “category,”  there  is  potential  for  further 
differentiation  by  “type.”  For  example,  the  cat¬ 
egory  of  “Repair  of  Civic  and  Cultural  Facilities” 
includes  such  disparate  types  as  mosque  construc¬ 
tion  and  telecommunication,  radio,  and  support 
services.  Projects  are  also  varied  in  specified 
desired  effects,  a  point  to  be  discussed  in  depth 
later.  This  degree  of  variance  in  combination 
with  the  significant  number  of  CERP  projects 
leads  to  the  second  cautionary  note:  findings 


no  comprehensive  study  of  CERP  in 
which  DOD  could  have  confidence  is 
possible  using  only  unclassified  data 

regarding  CERP  effectiveness  drawn  from  anec¬ 
dotes  or  case  studies  alone  should  be  viewed  as 
a  weak  basis  from  which  to  draw  general  conclu¬ 
sions.  In  addition,  studies  that  have  looked  at 
thousands  of  cases  but  treated  CERP  projects 
indiscriminately — by  aggregating  all  projects  by 
cost,  number,  province,  or  country — may  not 
serve  the  community  well  since  they  also  fail  to 
account  for  the  variance  described  above. 

Accounting  for  the  Context 

CERP  is  being  employed  in  a  complex  and 
dynamic  environment.  Measuring  effectiveness 
in  such  an  environment,  given  the  multitude 
of  extraneous  factors  that  may  influence  the 
achievement  of  a  desired  effect,  is  challeng¬ 
ing  but  not  impossible.  This  is  where  a  strong 
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research  approach  and  rigorous  research  designs 
come  into  play. 

CERP  is  not  used  in  a  laboratory  setting 
in  a  way  that  one  could,  as  in  a  true  experi¬ 
ment  design,  control  for  the  potential  influ¬ 
ence  of  specific  extraneous  factors  by  physi¬ 
cally  excluding  them  from  the  environment.  It 
is  instead  used  in  an  environment  where  other 
factors  such  as  high-tempo  operations  may  or 
may  not  be  taking  place,  coalition  forces  may 
or  may  not  be  present,  ethnic  or  tribal  bound¬ 
aries  may  or  may  not  be  crossed,  projects  may 
or  may  not  have  been  consistent  with  com¬ 
munity  leaders’  priorities,  good  governance 
may  be  present  in  one  locale  but  not  another, 


there  are  as  many  perspectives  on 
the  desired  effect  of  CERP  employment 
as  there  are  conclusions  regarding 
CERP  effectiveness 


and  so  on.  Many  of  these  data  are  classified, 
serving  as  yet  another  example  of  the  impor¬ 
tant  role  classification  plays  in  CERP  analyses. 
Comprehensive  analyses  of  CERP  effective¬ 
ness  must  take  the  potential  influences  of  these 
other  factors  into  account.  This  can  be  done 
systematically  in  several  ways,  two  of  which 
are  described  below. 

In  one  type  of  comparative  method,  the 
research  design  controls  for  extraneous  factors 
by  comparing  cases  that  share  the  same  or  simi¬ 
lar  values  for  the  factors  (Most  Similar  Systems 
Design)  but  differ  on  the  value  of  the  dependent 
variable  (was  the  effect  achieved  or  not?)  on 
the  assumption  that  doing  so  increases  confi¬ 
dence  that  changes  in  the  dependent  variable 
are  associated  with  changes  in  the  independent 
variable.  For  example,  were  one  to  identify  two 
cases — one  in  which  CERP  was  employed  and 


one  where  it  was  not — where  the  extraneous 
factors  in  both  were  similar  in  many  respects 
and  the  desired  effect  was  achieved  in  the  case 
where  CERP  was  employed  and  not  achieved  in 
the  other  case,  one  could  have  some  confidence 
that  the  effect  was  a  consequence  of  the  CERP 
project  and  not  other  extraneous  factors. 

While  this  research  design  would  increase 
confidence  that  CERP  effectiveness  was  actually 
being  measured,  it  has  limitations.  The  two-case 
focus  decreases  confidence  that  the  measured 
effectiveness  can  be  generalized  to  the  larger 
project  population.  A  different  research  design, 
one  that  could  account  for  context  and  increase 
confidence  in  the  generalization  of  results,  is  the 
statistical  method. 

In  the  statistical  method,  extraneous  factors 
are  not  controlled  for;  rather,  they  are  measured 
and  their  influence  is  accounted  for  through 
statistical  analysis.  If  a  well-designed  statistical 
model  looking  at  thousands  of  cases  concluded 
that  CERP  had  been  associated  with  changes 
in  the  desired  effect  in  a  statistically  significant 
manner,  confidence  that  an  effect  can  be  gen¬ 
eralized  would  be  warranted. 

Given  that  different  research  designs  have 
different  strengths  and  weaknesses,  a  research 
approach  to  studying  the  effectiveness  of  CERP 
should  be  multimethod  in  character.  Robust 
findings  across  multiple  methods  would  increase 
confidence  in  conclusions  regarding  effective¬ 
ness.  In  addition,  as  often  occurs  in  research, 
findings  from  one  method  will  tend  to  suggest 
important  questions  that  require  additional 
analyses  using  a  variant  of  the  same  method  or 
another  method  altogether. 

Understanding  the  Desired  Effect 

That  Which  Has  Been  Claimed.  There 
are  as  many  perspectives  on  the  desired  effect 
of  CERP  employment  as  there  are  conclusions 
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regarding  CERP  effectiveness.  Analysts  have 
claimed  it  is  intended  to  improve  security 
through  the  purchase  of  loyalty  or  informa¬ 
tion,  improve  governance,  protect  forces,  build 
capacity,  or  improve  the  relationship  between 
the  Afghan  population  and  government  of  the 
Islamic  Republic  of  Afghanistan.7  U.S.  Forces- 
Afghanistan  (USFOR-A)  guidance  on  CERP 
as  a  weapons  system  instructs  commanders  to 
employ  it  in  order  to  build  capacity,  promote 
peace  and  hope  for  future  generations,  and 
build  trust  and  lasting  support  for  the  Afghan 
government.8  Other  Regional  Command-East 
guidance  uses  similar  phraseology  and  adds  that 
CERP  projects  enhance  economic  opportunities 
as  a  viable  alternative  to  the  insurgency.9 

These  desired  effects  run  the  gamut  from 
the  tactical  to  the  strategic  level  of  war.  CERP 
certainly  could  be  characterized  as  a  tactical 
weapons  system  capable  of  generating  tactical, 
operational,  and  strategic  effects.  From  an  ana¬ 
lytical  perspective,  this  is  an  important  issue,  as 
the  focus  of  and  research  design  for  measuring 
effectiveness  at  the  three  levels  would  be  differ¬ 
ent.  The  lack  of  clarity  on  the  level  of  analysis 
in  the  CERP  debate  has  made  a  large  contribu¬ 
tion  to  its  inconclusive  nature. 

Given  the  wide  range  of  CERP  project 
categories,  it  is  conceivable  that  each  of  the 
desired  effects  listed  is  valid  in  conjunction 
with  one  or  more  of  those  categories  or  combi¬ 
nations  thereof.  Indeed,  an  example  of  this  from 
our  own  research  is  offered  later.  However,  and 
to  repeat,  there  is  no  guidance  specifying  such 
relationships  and  no  research  that,  as  yet,  pro¬ 
vides  empirical  support  for  them.  Without  an 
understanding  of  the  desired  effect  (were  CERP 
to  be  treated  in  the  aggregate)  or  of  certain 
effects  associated  only  with  certain  categories  of 
expenditures,  little  progress  can  be  made  toward 
evaluating  CERP  effectiveness.10 


This  plethora  of  perspectives  is  impor¬ 
tant  to  consider  from  another  angle:  train¬ 
ing.  Without  a  clearly  specified  mechanism  of 
action  and  desired  effect,  commanders  cannot 
be  trained  to  understand  well  and  plan  against 
the  effectiveness  of  this  weapons  system  at  any 
level  of  analysis.  The  absence  of  a  comprehen¬ 
sive  training  program  puts  warfighters  at  a  dis¬ 
advantage  and  at  risk. 

That  Which  Has  Been  Analyzed.  Ideally, 
assessing  effectiveness  should  be  done  after  a 
desired  effect  and  a  concomitant  direct  measure 
of  effectiveness  have  been  identified.  Absent  a 
direct  measure,  indirect  and  surrogate  measures 
can  be  pursued.  Agreement  on  a  desired  effect 
is  far  from  an  apt  description  of  the  current  state 
of  affairs.  Nonetheless,  in  analyses  of  CERP 
(treated  in  the  aggregate),  the  direct  measure  of 
effectiveness  most  often  used  has  been  changes  in 
levels  of  violence.  The  hypothesis  is  that  if  CERP 
were  effective,  it  would  manifest  as  reductions 
in  violence.  This  hypothesis  likely  follows  from 
several  analysts’  claims  that  the  desired  effect  is 
improved  security.  The  availability  of  unclas¬ 
sified  data  may  also  have  played  a  role  in  this 
choice — improvised  explosive  device  (IED) 
events,  for  example,  can  be  downloaded  from 
CIDNE  in  an  unclassified  format. 

Does  the  measure  changes  in  levels  of  vio¬ 
lence  serve  well  as  a  valid  direct  measure  of 
effectiveness?  It  is  insightful  to  tally  how  often 
it  is  listed  as  a  desired  effect  by  CERP  prac¬ 
titioners.  In  a  review  of  over  2,000  CIDNE 
CERP  records  from  Afghanistan,  less  than 
10  percent  specified  a  primary  or  secondary 
intended  benefit  that  could  reasonably  be 
equated  with  changes  in  levels  of  violence, 
with  the  intended  change  being  a  reduction.11 
This  should  give  one  pause,  and  it  highlights 
the  third  cautionary  note  with  regard  to  recent 
studies:  the  conclusions  being  drawn  about 
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effectiveness  may  not  be  valid  as  they  may  not 
speak  directly  to  the  effects  specified  by  the 
initiators  of  the  projects  under  analysis. 

It  may  be  that  this  clear  disconnect  lies 
in  the  levels  of  analysis  issue  raised  previously. 
Perhaps  the  approximately  10  percent  specified 
by  the  practitioners  are  desired  tactical  effects 
and  the  recent  studies  were  more  strategically 
focused.  A  strategic-level  study  of  the  effective¬ 
ness  of  CERP  using  changes  in  levels  of  violence 
as  the  direct  measure  of  improved  security  should 
take  into  account  other  factors  that  may  be  asso¬ 
ciated  with  such  changes,  the  most  significant 
of  which  is  the  presence  of  U.S.  and  coalition 
forces.  No  such  rigorous  accounting  has  been 
seen  in  recent  studies.  So  again,  one  should  give 
pause  when  considering  the  conclusions. 


just  as  care  should  be  taken  when 
considering  analyzing  CERP  in  the 
aggregate,  so  too  should  care 
accompany  approaches  to  treating 
violence  in  the  aggregate 


Does  the  changes  in  levels  of  violence  mea¬ 
sure  serve  well  as  a  valid  indirect  measure  of 
effectiveness?  Indirect  measures  are  useful  when 
gauging  a  direct  effect  poses  a  significant  chal¬ 
lenge,  and  they  are  valid  if  they  correlate  with 
direct  measures.  Since  there  is  no  agreement 
on  the  valid  direct  measure,  analysts  can  only 
speculate  as  to  whether  it  may  correlate  with 
changes  in  levels  of  violence.  But  why  bother 
doing  that  when  several  of  the  direct  measures 
specified  by  practitioners  are  measurable?  For 
example,  the  effectiveness  of  projects  intended 
to  increase  activity  at  a  market  or  bazaar  could 
be  measured  by  analyzing  Ground  Moving 
Target  Indicator  (GMTI)  data  on  access  roads 


to  those  locales  in  periods  before  and  after  proj¬ 
ect  completion.  As  one  might  expect,  those 
data  are  classified. 

Are  changes  in  levels  of  violence  a  reason¬ 
able  surrogate  measure?  Surrogate  measures  are 
useful  when  issues  such  as  time-sensitivity  or 
cost  prohibit  waiting  until  a  desired  effect  is 
measurable.  They  are  often  referred  to  as  path- 
dependent  measures  whose  values  correlate  with 
those  of  the  desired  effects.  Similar  to  what  was 
stated  above,  as  there  is  no  agreed-upon  per¬ 
spective  regarding  the  mechanism  of  action  of 
CERP,  there  is  no  way  at  this  time  of  determin¬ 
ing  if  changes  in  levels  of  violence  is  a  valid 
surrogate  measure  of  CERP  effectiveness. 

Finally,  just  as  care  should  be  taken  when 
considering  analyzing  CERP  in  the  aggregate, 
so  too  should  care  accompany  approaches  to 
treating  violence  in  the  aggregate.  The  dif¬ 
ference  between  Afghan-on- Afghan  violence 
vis-a-vis  other  types  can  be  an  important  dis¬ 
criminator  in  understanding  the  effectiveness 
of  CERP  projects. 

That  Which  Requires  Qreater 
Consideration.  Our  experiences  suggest  that 
several  additional  nuances  should  be  brought 
to  bear  in  analyses  of  CERP’s  effectiveness.  The 
first  is  found  in  a  careful  reading  of  USFOR-A 
guidance  on  CERP  employment;  CERP  is 
intended  for  projects  that  can  be  sustained  by 
the  local  population  or  government,  and  com¬ 
manders  need  to  evaluate  how  projects  can  add 
value  to  the  local  community.12  This  empha¬ 
sis  on  local  suggests  that  the  measurement  of  a 
desired  effect  should  be  focused  in  an  area  prox¬ 
imate  to  that  in  which  the  CERP  project  was 
completed.  For  projects  where  the  desired  effect 
may  be  to  create  or  restore  goodwill  or  suppress 
ill-will  (for  example,  from  a  battle  damage  pay¬ 
ment),  this  notion  also  suggests  an  interesting 
sensitivity  analysis  that  could  explore  degrees  of 
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decay  in  effectiveness  as  a  function  of  proximity 
to  the  project. 

A  second  nuance  is  the  importance  of 
the  temporal  domain.  Many  of  the  positive 
effects  that  may  be  generated  by  CERP  proj¬ 
ects  likely  fade  over  time — but  over  what  time 
period,  and  to  what  degree?  At  what  point  in 
time  after  project  completion  is  an  effect  most 
pronounced,  and  at  what  point  does  it  begin  to 
diminish?  Sensitivity  analyses  focusing  on  decay 
over  time  could  answer  these  questions  and 
could  inform  a  commander’s  decisions  regard¬ 
ing  the  sequencing  of  activities.  Of  course,  one 
could  also  perform  sensitivity  analyses  on  com¬ 
binations  of  time,  proximity,  and  project  type. 

A  third  nuance,  tightly  coupled  to  the  first, 
is  the  recognition  that  not  all  violence  is  local. 
If  reduction  in  violence  is  the  desired  effect  of  a 
project  that  was  in  an  area  proximate  to  a  main 
supply  route,  including  any  IEDs  or  other  vio¬ 
lent  events  along  the  route  in  the  analysis  may 
be  inappropriate. 

The  more  one  studies  CERP,  the  more  one 
comes  to  understand  the  nuances  that  should  be 
taken  into  account  when  drawing  conclusions 
regarding  its  effectiveness. 

Reasons  for  Optimism 

Over  the  past  year,  we  have  steadily  and 
patiently  built  the  foundation  for  executing 
studies  of  CERP  that  satisfy  the  demanding 
criteria  put  forth  in  this  article.  The  base  of 
that  foundation  is  data.13  It  has  been  a  signifi¬ 
cant  undertaking  getting  access  to,  gathering, 
reviewing,  and  cleaning  all  of  the  data  required 
to  execute  analyses  that  satisfy  the  minimal 
criteria  specified  above.  We  are  grateful  to  the 
Joint  Advanced  Warfighting  Program  for  spon¬ 
soring  that  work  and  to  the  warfighters  who 
have  taken  the  time  to  share  data.  Creating  the 
data  foundation  is  not  as  simple  as  downloading 


information  from  CIDNE.  It  requires  a  review 
of  brigade  battlefield/commander  update  assess¬ 
ments,  patrol  reports,  Human  Terrain  Team 
reports,  U.S.  (and  other  coalition)  force  track¬ 
ing  data,  GMTI  data,  and  Afghanistan  National 
Security  Force  development  reports.  It  also  calls 
for  interviews  with  brigade,  battalion,  and  com¬ 
pany  commanders.  Finally,  it  requires  cleaning 
and  translating  all  these  data  into  a  format 
suitable  for  analysis.  We  have  reciprocated  that 
cooperation  by  communicating  findings  back  to 
commanders  who  have,  in  turn,  requested  addi¬ 
tional  analyses  in  order  to  better  inform  their 
resourcing  decisions.  One  such  study  and  addi¬ 
tional  findings  are  presented  below,  albeit  in  a 
way  that  takes  account  of  classification  issues. 


we  identified  a  positive  association 
between  culvert-denial  systems  and  a 
subsequent  reduction  in  IED  events 

Study:  Culvert-denial  Systems 

A  deployed  unit  requested  an  analysis  of 
the  effectiveness  of  culvert-denial  systems. 

Many  roads  in  Afghanistan  are  undercut  by 
culverts  that  present  opportunities  to  insur¬ 
gents  seeking  to  intimidate  or  disrupt  free¬ 
dom  of  movement  of  the  Afghan  population. 
Culverts  are  ideal  for  the  emplacement  of 
high  net  explosive  weight  (HNEW)  IEDs  as 
they  are  ubiquitous,  ample  (allowing  for  sig¬ 
nificantly  sized  IEDs),  and  accessible  (allowing 
for  repeated  visits  over  which  to  build  an  IED). 

When  a  vehicle  encounters  a  culvert-emplaced 
HNEW  IED,  the  immediate  outcome  often 
includes  a  destroyed  vehicle,  casualties,  and 
road  debris.  The  effects  of  such  an  event  could 
include  reduced  freedom  of  movement  of  the 
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Afghan  population,  which  could,  in  turn, 
impact  commerce  and  a  host  of  other  issues. 

Research  Design,  Context,  Desired  Effect, 
and  Findings.  Culvert-denial  system  projects 
are  a  subtype  that  appears  in  several  different 
CERP  categories.  After  a  thorough  review  of  the 
three  CERP  databases  described  previously,  sev¬ 
eral  hundred  such  projects  were  identified.  This 
large  number  and  the  desire  to  test  for  findings 
that  can  be  generalized  suggested  the  statisti¬ 
cal  method.  The  comparative  method  was  also 
brought  to  bear  on  the  question. 

Culvert-denial  systems  have  been 
employed  in  many  different  locations  through¬ 
out  Afghanistan,  crossing  district,  provincial, 
ethnic,  and  tribal  boundaries;  in  areas  that  are 
densely  and  sparsely  populated;  and  in  areas 
where  the  coalition’s  operations  tempo  has 
been  both  high  and  low.  A  statistical  model  was 
designed  to  identify  the  influence  that  such  fac¬ 
tors  may  have  on  whether  installing  a  culvert- 
denial  system  produces  the  desired  effect. 

What  is  the  desired  effect?  Since  these  sys¬ 
tems  are  employed  to  deny  insurgents  the  abil¬ 
ity  to  hastily  emplace  IEDs  in  culverts  that  can 
injure  or  kill  the  Afghan  population,  the  direct 
measure  of  effectiveness  is  changes  in  levels  of 
IED  events  on  or  immediately  proximate  to  culverts. 
This  was  measured  using  IED  data  exported  from 
CIDNE  for  a  defined  period  preceding  and  fol¬ 
lowing  the  installation  of  culvert-denial  systems. 

In  reviewing  the  data  on  culvert-denial  sys¬ 
tems,  an  opportunity  for  additional  analysis  was 
identified  for  which  the  comparative  method 
served  well.  There  was  a  period  during  which 
a  commander  had  units  of  similar  capability 
dispersed  over  similar  terrain  (both  in  terms  of 
terrain  features  and  population  density)  that 
included  an  80-kilometer  (km)  stretch  of  road. 
The  first  40  km  of  this  road  had  culvert  denial 
systems  installed  and  the  second  half  did  not, 


as  the  commander  was  waiting  to  receive  fund¬ 
ing  for  it. 

Using  the  statistical  method,  we  identified 
a  statistically  significant  positive  association 
between  the  introduction  of  culvert-denial  sys¬ 
tems  and  a  subsequent  reduction  in  IED  events 
on  or  near  the  culverts.  In  the  comparative  case 
study,  no  IED  explosions  were  reported  for  the 
40  km  of  road  on  which  culvert-denial  systems 
were  installed  versus  five  IED  explosions  for  the 
40  km  of  road  where  they  were  not  installed. 
Upon  completion  of  these  studies,  the  findings 
were  swiftly  communicated  back  to  the  field. 

This  analysis  could  be  characterized  as  a  tac¬ 
tical  analysis  of  effectiveness.  Emplacing  these 
systems  can  also  have  operational  and  strategic 
effects.  Follow-on  operational  and  strategic-level 
analyses  will  focus  on  measuring  changes  in  free¬ 
dom  of  movement  on  these  roads  (using  GMTI 
data)  in  addition  to  other  measures. 

Cost  versus  Value.  USFOR-A  guidance 
regarding  CERP  instructs  that  CERP  will  not 
be  used  for  “direct  or  indirect  benefit  to  U.S., 
Coalition,  or  other  supporting  military  person¬ 
nel.”14  However,  that  does  not  prevent  the  ana¬ 
lytic  community  from  assisting  DOD  in  consid¬ 
ering  direct  or  indirect  benefits  to  U.S.  forces 
for  purposes  of  calculating  value.  The  culvert- 
denial  system  analyses  can  be  used  to  illustrate 
what  is  possible. 

U.S.  forces,  mounted  and  dismounted,  cross 
culverts  with  frequency  equal  to,  if  not  greater 
than,  the  Afghans.  Insurgents  are  aware  of  this, 
and  are  also  aware  that  a  culvert  provides  one 
of  few  opportunities  to  emplace  a  HNEW  IED 
that  is  capable  of  disabling  or  destroying  a  Mine 
Resistant  Ambush  Protected  (MRAP)  vehicle. 
A  number  of  different  systems  are  used  to  deny 
insurgents  access  to  culverts,  but  for  purposes 
of  illustration,  the  cost  of  a  cage  constructed  of 
reinforcing  bar  (rebar)  is  approximately  $5,000 
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per  unit.  There  are  many  variants  of  MRAPs  whose  unit  cost  can  range  from  $500,000  to  over  a 
million  dollars.  A  simple  material-based,  cost-versus-value  calculation  suggests  that,  in  reference  to 
MRAP  survival,  one  rebar  system  has  a  value  ranging  between  100  and  200  times  its  cost. 

Additional  Findings 

The  culvert-denial  system  analyses  demonstrate  a  value  of  analyzing  CERP  in  a  far  more  dis¬ 
criminate  manner  than  has  generally  been  done.  The  findings  were  of  immediate  use  to  warfighters 
in  Afghanistan  and  offer  a  concrete  example  to  policymakers  of  how  CERP  can  significantly  impact 
operations.  We  have  continued  to  receive  requests  for  analyses  from  the  field  and  in  the  course  of 
research  have  identified  a  number  of  intriguing  findings  that  speak  to  the  salience  of  nuance  raised 
in  this  article. 

Terrain  Features.  Aspects  of  the  operational  environment  are  proving  to  be  important  factors 
to  consider  in  CERP  analyses.  It  was  suggested  previously  that  effects  should  be  measured  proximate 
to  a  project.  Based  on  analyses  of  completed  CERP  projects  in  both  urban  and  rural  environments, 
we  encourage  analysts  to  consider  terrain  features  when  setting  the  distance  about  a  project  in  which 
to  measure  for  the  desired  effect.  We  have  discovered  that  urban  terrain  features  appear  to  limit 
the  geographic  reach  of  a  desired  effect.  If  we  cast  the  net  too  far  from  an  urban  project,  evidence 
of  an  effect  can  be  overwhelmed  by  unrelated  activity  at  the  periphery  of  the  area  of  analyses.  This 
finding  is  quite  consistent  with  the  observations  offered  in  joint  doctrine  on  military  operations  in 
urban  terrain.15 
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Types  of  Projects  and  Types  of  Violence. 

In  one  province  under  analysis,  a  relationship 
was  identified  between  economic  development 
projects  and  increased  proximate  levels  of  vio¬ 
lence.  This  was  not  the  desired  effect  of  these 
projects,  of  course,  so  the  finding  was  trou¬ 
bling.  Once  identified,  it  would  have  been  of 
no  use  to  warfighters  to  share  this  finding  and 
then  simply  walk  away,  leaving  them  with  the 
impression  that  economic  development  proj¬ 
ects  invited  violence.  Identifying  the  types  of 
projects  and  type  of  violence  proved  insight¬ 
ful  in  revealing  an  underlying  dynamic — a 
particular  set  of  projects  was  fueling  a  turf 


greater  access  is  a  call  for  the 
development  and  execution  of  a 
comprehensive  data-gathering  strategy 
so  a  better  understanding  of  nonkinetic 
capabilities  can  be  developed 

war  among  local  powerbrokers  resulting  in 
Afghan-on- Afghan  violence.  This  level  of 
detail  is  of  value  to  commanders  as  it  suggests 
the  motivations  of  the  powerbrokers;  in  turn, 
that  may  offer  a  means  through  which  to  man¬ 
age  their  participation  in  the  counterinsur¬ 
gency  more  effectively. 

Operational  and  Strategic-level  Analyses 
Interaction  Effects.  The  culvert-denial  system 
comparative  case  study  controlled  for  the  pres¬ 
ence  of  U.S.  forces  by  selecting  a  period  when 
units  in  both  cases  were  of  comparable  capability 
and  patrolled  a  space  of  comparable  geographic 
scope  both  before  and  after  the  installation  of 
culvert-denial  systems.  That  is,  U.S.  presence 
was  essentially  constant.  Our  most  important 
findings  from  a  strategic  and  operational  per¬ 
spective  are  emerging  from  analyses  of  projects 


in  which  U.S.  presence  proximate  to  a  project 
actually  changed  after  project  completion. 

This  research  line  of  effort  focused  on  test¬ 
ing  for  a  strategic-  or  operational- level  effect 
and  assumed  the  desired  effect  was  improved 
security.  The  effect  was  operationalized  as 
changes  in  level  of  violence.  CERP  projects 
were  not  treated  in  toto  since  our  other  research 
identified  significant  differences  in  effectiveness 
across  categories.  Instead,  projects  from  several 
“like”  categories  were  bundled  and  those  bun¬ 
dles  were  used  as  the  units  of  analysis. 

As  mentioned,  any  analysis  of  CERP  proj¬ 
ects  in  which  changes  in  level  of  violence  is  the 
desired  effect  must  take  into  account  other  fac¬ 
tors  that  could  have  an  impact  on  violence.  The 
most  significant  of  those  factors  is  the  presence 
of  U.S.  forces  proximate  to  a  project.  Changes 
in  presence  were  measured  by  analyzing  Blue 
Force  Tracking  data. 

Analyses  have  led  us  to  conclude  that  there 
is  an  important  interaction  effect  associated  with 
changes  in  levels  of  violence  in  the  proximate 
area  where  a  project  is  completed  and  changes  in 
the  density  of  Blue  Forces.  Increases  or  decreases 
in  presence  nearby  a  project  are  systematically 
and  significantly  associated  with  the  changes  in 
levels  of  violence  and  the  degree  of  the  changes. 
This  interaction  effect  is  significant  in  analyses  of 
several  bundles  of  CERP  projects,  during  different 
phases  of  operations,  and  across  different  locales 
in  Afghanistan.  It  is  not  an  understatement  to 
describe  the  interaction  effect  as  substantial  for 
certain  bundles  of  CERP  expenditures  when  com¬ 
pared  to  cases  in  which  changes  in  U.S.  presence 
occurred  in  areas  where  projects  may  or  may  not 
have  been  completed.  (A  random  sample  of  loca¬ 
tions  was  analyzed  to  establish  a  baseline  against 
which  to  compare  the  interaction  effect.) 

Should  this  relationship  be  robust  in  the 
face  of  updated  data  sets  (both  longitudinally 
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and  in  richness),  it  could  advance  the  concep¬ 
tual  development  of  combined  arms  (or,  more 
appropriately,  combined  capabilities)  warfare  for 
counterinsurgency  operations.  Its  more  immedi¬ 
ate  value,  however,  from  both  an  operational  and 
strategic  perspective  could  be  as  an  input  into  the 
calculus  behind  decisions  regarding  how  best  to 
surge  in  an  area — by  sequencing  or  synchronizing 
kinetic  and  nonkinetic  capabilities — or  whether 
and  how  best  to  “thin  out”  for  purposes  of  transi¬ 
tioning  to  a  host  nation  government. 

Conclusion 

The  debate  regarding  the  effectiveness  of 
CERP  has  been  fueled  by  studies  that  may  rea¬ 
sonably  be  characterized  as  incomplete  since 
they  do  not  address  satisfactorily  the  sev¬ 
eral  issues  identified  in  this  article.  This  dis¬ 
course,  as  does  any  that  is  not  well-supported 
by  evidence,  carries  risk — in  this  case  to  the 
warfighter  employing  the  capability  and  the 
policymaker  determining  whether  to  fund  it. 
Such  risks  can  be  reduced  by  a  sustained  effort 
to  better  understand  and  then  gather  the  data 
required  to  support  rigorous  analyses  of  CERP 
and  other  nonkinetic  activities.  That  said,  fur¬ 
ther  increasing  the  burden  on  unit  command¬ 
ers  to  record  even  more  data  is  not  a  course  of 
action  advocated  here. 

For  capabilities  already  fielded,  there 
is  plenty  of  data  out  there:  it  takes  only  one 
trip  from  a  company  headquarters  to  its  bat¬ 
talion  headquarters  to  its  brigade  headquar¬ 
ters  to  its  regional  command  headquarters  to 
International  Security  Assistance  Force  (ISAF) 
joint  command  headquarters  and  to  ISAF  head¬ 
quarters  to  be  made  patently  aware  of  that  fact. 
That  mass  of  data,  of  course,  is  not  “clean,”  is 
in  many  formats,  and  tends  not  to  be  consistent 
across  units.  As  noted  at  the  outset  of  this  arti¬ 
cle,  it  is  a  challenging  environment  for  analysts, 


and  the  analytical  community  has  quite  a  bit 
more  work  ahead  in  that  regard.  With  greater 
access  to  these  data  and  strong  research 
approaches  and  designs,  improved  analyses  in 
which  DOD  and  others  can  have  increased 
confidence  are  possible.  Greater  access  is  not 
a  call  for  prematurely  declassifying  much  of  the 
data  required  for  thorough  analyses.  It  is  a  call 
for  the  development  and  execution  of  a  com¬ 
prehensive  data-gathering  strategy  so  a  better 
understanding  of  these  nonkinetic  capabilities 
can  be  developed. 

The  findings  presented  here  demonstrate 
that  committing  resources  to  this  analytical  task 
can  have  substantial  returns  in  answering  ques¬ 
tions  regarding  CERP  effectiveness.  A  final  note 
of  caution  is  in  order,  however.  Such  questions 
should  be  thoughtfully  crafted  and  informed  by 
the  issues  and  nuances  identified  in  this  article. 
A  response  to  a  simple  query  regarding  CERP 
effectiveness  will  provide  little  if  any  insight  to 
the  one  asking  the  question.  Well-structured 
queries  that  take  into  account  the  nuances  dis¬ 
cussed  previously  will  result  in  far  more  infor¬ 
mative  responses. 

The  analytical  challenges  presented  here 
come  from  having  to  analyze  CERP  effective¬ 
ness  in  a  nonpermissive  environment  long  after 
its  employment.  In  the  future,  to  better  under¬ 
stand  such  capabilities  before  they  are  employed 
and  to  establish  an  evaluation  regime,  the  first 
task  is  to  step  back  and  consider  what  effects 
are  desired,  what  mechanisms  of  action  exist 
for  their  success  or  failure,  and  then  and  only 
then  what  data  should  be  captured  to  deter¬ 
mine  operational  effectiveness.  This  typical 
preemployment  operational  test  and  evalua¬ 
tion  approach  for  kinetic  capabilities  should  be 
made  typical  for  nonkinetic  capabilities  as  well, 
so  they  can  be  employed  in  the  most  operation¬ 
ally-  and  cost-effective  manners.  PRISM 


PRISM  2.  NO.  4 


LESSONS  LEARNED  I  149 


FISCHERKELLER 


Notes 
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Enduring  Freedom  and  Iraqi  Freedom  (for  example,  biometrics  and  forensics,  U.S.  Agency  for  International 
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3  There  is  no  formal  definition  of  Quick  Reaction  Capability.  It  has  entered  Defense  Department  discourse 
as  a  means  of  describing  a  capability  fielded  in  response  to  an  urgent  operational  needs  statement. 

4  Administrative  guidance  for  the  Commander’s  Emergency  Response  Program  (CERP)  funds  is  found 
in  “Money  as  a  Weapons  System  Afghanistan,”  U.S.  Forces-Afghanistan  Publication  1-06,  Commanders 
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